Identification of potentially cancerous lesions in CT and MRI studies is a common task of radiologists. Once such a lesion has been identified, it is desirable to determine information pertaining to the lesion, such as the extent and the volume such a lesion occupies and other descriptors. The radiologist's report documents the findings of the examination. Part of this report contains the location of the lesion and this is usually reported relative to anatomical structures or landmarks. The use of anatomical landmarks is preferred as the basic anatomy is common to all patients and is independent of the particular examination conducted.
The visual interpretation of the examination by the radiologist can be done in conjunction with other aids, such as computer assisted diagnosis (CAD) software, but the radiologist makes the final interpretative decisions. Once a radiologist has determined a lesion represents an abnormal condition and is to be noted, it is possible for automated image processing tools to estimate information about the lesion, such as the volume it occupies. A common method is for the radiologist to initiate the tools by pointing to the lesion with a mouse or similar pointing device. The results, such as the volume, need to be included in a report.
A further important descriptor of a lesion consists of its location in the body. Typically, location will be specified by reference to anatomical structure or structure sub-division. A distinct and standard system of nomenclature may be associated with different lesion types and different organs or systems in the human body. For example, the location of lesions in the lungs will typically be specified with identification of the left or right lung and the lung lobe in which the lesion resides. Lesions in the liver can be located with respect to a standard classification system for the liver lobes, or with respect to the branching hierarchy of the portal vein network.
The example of a lesion located in one of the lungs illustrates the type of specific locating terminology that might be used in a medical report. For the purpose of the description herein, the terms lung lesion and lung nodule are synonymous and should be considered interchangeable. The two lungs are usually comprised hierarchically of five lobes, three lobes on the right-hand side and two on the left-hand side. Each lobe comprises an independent unit from the viewpoints of function and anatomical structure. Thus, each lobe receives a distinct main branch of the pulmonary arterial and venous circulation, as well as of the airways of the bronchial tree. The lobes are also spatially distinct, with the two lobes on the left side being an upper and lower lobe and the three lobes on the right side forming an upper, middle and lower region.
The hierarchical structure of the lung lobes is reflected in the branching, tree-like architecture of the pulmonary vasculature and bronchial airways. The pulmonary arteries form a tree structure with the root of the tree being the right ventricle of the heart. The vessel tree enters the lungs in the central thoracic region called the mediastinum. Two branches of the tree supply depleted blood to the left-hand side of the lungs and three branches supply the right-hand side. The five main branches of the pulmonary arteries determine the five lobes of the lung. The arteries continue to divide into vessels with smaller diameters in order eventually to perfuse the alveolar structures in the lung with an ample supply of blood to be cleansed of waste gases and infused with oxygen. The bronchial structure of the lung is similar and runs parallel to the vascular structure. A demarcation of the lobes is sometimes visible in the CT examination as a faint line, called a fissure. In addition, sometimes the lobes of the lung are further defined into anterior or posterior locations within the lobe. The more refined the description, the easier it is for a subsequent radiologist or clinician to relocate the lesion noted.
It is desirable for the radiologist to point to the lesion and have the information about the lesion and the location of the lesion to be automatically reported into the patient report. Requiring the radiologist to transcribe the results for placement into the report is a source of error and also requires additional time. For this reason, it is desirous to report the location of the lesion automatically and in terms commonly used. However, the automatic reporting of the lesion location must perform the task at least as fast as manual methods. If processing times are longer than the manual approach, then the advantage of the automatic method is lost and the automatic approach becomes impractical.
One method to determine the lobe containing a lesion is to segment the lungs into five zones and determine the zone in which the lesion is located. A number of methods to accomplish this task exist. For example, see X. Zhou, et al, “Automatic segmentation and recognition of anatomical structures from high-resolution CT images,” Computerized Medical Imaging and Graphics, Vol. 30, pp. 299-313, 2006. However, these methods require significant computational resources and are inefficient. To perform lung lobe segmentation, all the lung voxels are classified according to the lobe to which they belong. However, to identify the lobar location of a lung lesion, there is only a need to perform the labeling for a single or small number of voxels at the location of the lesion, and so the additional labeling effort is not utilized. The embodiments of this invention are intended to address the problem of identifying the lung lobe of a specific voxel and not the lobar segmentation of the entire lung. While in theory the method would produce complete lung lobar labeling, it would however lead to an inferior result in terms of overall computational efficiency.
Both the vascular and bronchial structures of the lungs take the form of a tubular vessel tree branching network. Means of identifying tubular structures are well-known (see F. Frangi, et al., “Multiscale vessel enhancement filtering,” in W. Wells, A. Colchester, and S. Delp, editors, MICCAI'98, volume 1496 of LNCS, pages 130-137. Springer-Verlag, Germany, 1998). These describe a tubular structure in mathematical terms and utilize the form and structure of the associated Hessian matrix of the CT image data. One result of the analysis of a potential tubular structure is a measure, commonly referred to as vesselness. Variations on the general notion of vesselness have been proposed, though in general all the variations utilize the eigenvalues and eigenvectors of the Hessian matrix in some manner. We recognize a voxel can be identified as being part of a tubular structure using these methods. However, the methods also require considerable computational resources and a wholesale use of such a measure throughout a CT examination is impractical.
Understanding the hierarchical structure and branching relationships of the vessels in a vasculature tree has been the object of many research efforts. One method of vessel segmentation and tracking is based upon a fast marching method, which is allied to level-set methods (see T. Deschamps, et. al., “Vessel segmentation and blood flow simulation using Level-sets and Embedded methods,” International Congress Series 1268, pp. 75-80, 2004.) A different method of vessel tree tracking relies on the recursive application of an adaptive segmentation algorithm in a cylindrical neighborhood that follows down the branching structure of the vessel tree (see Juerg Tschirren, et al., “Intrathoracic Airway Trees Segmentation and Airway Morphology Analysis From Low-Dose CT Scans,” IEEE Trans. Medical Imaging, 24(12), 1529-1539, 2005.). This method has the advantage of building up incremental knowledge of the tree structure in a breadth-first (level by level) order, so that the branching position and level are known at any point during the tracking process. Thus, the tracking can be terminated when a desired level or a desired set of a priori known anatomically-important vessels have been tracked. Furthermore, the method can be highly adaptive, as the changing characteristics of the local cylindrical neighborhood can be used to adjust tracking strategies and algorithm parameters.
Using these methods to perform a segmentation of the entire vascular structure is impractical in terms of computational cost. This impracticality can result in an inefficient use of computational resources and longer processing times, and more importantly in ineffective use of the radiologist time. Furthermore, since vessel tree structures inherently decrease in physical dimension as they undergo repeated subdivision, and due to the finite imaging resolution used during any medical imaging collection, the accuracy and reliability of the vessel tracking must correspondingly diminish as the tree structure is tracked to finer and finer structural levels. Therefore, it would be desirable to confine vessel tracking for purposes of anatomical understanding and labeling to the initial tree levels (i.e., those closest to the root of the tree) in order to ensure maximum possible reliability.
We understand tracking and segmenting part of a vessel tree structure can form a basis for anatomical labeling. Labeling of anatomical structures consists of associating to them standard names that have agreed meaning in the medical community of interest. For example, branches of the human lung airway system can be labeled trachea, right main bronchus, left main bronchus, etc. Often such a labeling step requires that matching be performed between vessel segments actually found in a medical image of interest and the segments in a preconfigured model or set of models that represent the anatomical forms that can be found in the human population (see Tschirren, Juerg, et al., “Matching and Anatomical Labeling of Human Airway Tree”, IEEE Trans. Medical Imaging, 24(12), pp 1540-1547). Anatomical labeling must exhibit tolerance to possible errors in the segmentation process and ambiguities in the range of available matches. Segmentation errors include, for example, spurious vessel branches due to slight vessel boundary deformations caused by image noise and artifacts, missing vessel segments, and missing branches. Matching errors or ambiguities can arise as a result of segmentation errors and also as a result of normal or abnormal anatomical variations in the human population. The difficulties encountered in the task of anatomical labeling provide further motivation to construct an automated labeling system that makes use of only the most reliable autonomously identified anatomical features. The embodiments of the current invention provide a solution to these difficulties. It makes use of a bi-directional vessel tracking strategy that confines anatomic-label matching to those parts of anatomy where it can be performed with the highest accuracy. Tracking of small and hence low-reliability vessels does not involve labeling or matching against an anatomy model. Rather, such tracking is employed solely to find a connection between reliably labeled anatomy and anonymous anatomy of the same vessel tree system. Some human anatomical organs are associated with more than a single vessel tree structure. Due to the nature of the systemic and pulmonary circulations, organs are supplied with parallel systems of arterial and venous blood-containing vessel trees. In the lungs, the bronchial airway tree provides yet a third distributional system whose anatomical form assumes that of a vessel tree. With a view towards the goal of automated location labeling of lesions, and with acknowledgement of the inherent tendency of medical image understanding algorithms to encounter failure situations, advantage can be had of the parallelism of the coterminous vessel tree structures. Thus, it would be most advantageous to first attempt to identify and segment the most reliably identifiable vessel tree structure and then perform lesion localization based on that structure. This strategy can be employed whenever it is the case that the parallel vessel trees are each associated in a predictable way with the organ's natural subdivisions, so that a clinically meaningful location label can result from any of the trees.
In the specific case of the lung and of CT imagery, the major branches of the bronchial tree can usually be reliably segmented and identified due to the high imaging contrast between the air-filled bronchial lumen and the surrounding bronchial wall, as well as the large size of the structures involved. For these reasons, we have determined recourse should first be had to using the bronchial vessel tree as an anatomical frame of reference for location identification. However, there may arise circumstances in which the use of the bronchial tree cannot successfully localize a lesion, most likely due to difficulty in tracing back from the lesion to the main bronchi, as described below, or due to pulmonary disease. In this case, a second attempt can be made at localization using the pulmonary arterial vessel tree. (In the case of CT and MRI imaging studies using contrast-enhancing injections into the blood stream, the arterial vessel tree will typically exhibit higher image contrast, and hence will enable more reliable automated segmentation, than will the pulmonary venous vessel tree.) Segmentation of the pulmonary tree can be accomplished by locating organs such as the pulmonary artery and the aorta (see Kitasaka, T., et. al., “Automated Extraction of Aorta and Pulmonary Artery in Mediastinum form 3D Chest X-ray CT Images without Contrast Mediuam,” SPIE Medical Imaging 2002, Vol. 4684, pp. 1496-1507.) Finally, should the use of the arterial tree also fail, a final attempt can be made to localize the lesion using the pulmonary venous vessel tree. Thus, an aspect of the advantages of the embodiments of the present invention over the current art resides in the ability to take advantage of the existence of inherent anatomical parallelism in the servicing vessel trees of human organs.
Alternatively, by referencing the feature to the branching structure of multiple hierarchical vessel trees, the location of an anatomical feature of interest, such as a lesion, may be identified separately and independently. Operating in this way permits advantage to be taken of the positional redundancy of the trees from the information-content point of view. Use of multiple trees, either automatically or under explicit control of an operator, can be used to increase the accuracy of localization and confidence in the result. Thus, an advantage of the embodiments of the present invention is to provide higher accuracy location information than is available in the current art.
Methods to track vessels have been investigated by numerous researchers and implementations of vessel tracking are part of many commercially available PACS systems. A common method for vessel tracking utilizes a so-called fast marching method. The fundamental idea is to use properties of vessels within the CT image stack as the basis to establish a tracking strategy. One such strategy is to consider a connected volume V, such as the lung, and suppose there is a function referred to as the potential function C: V→+ and furthermore suppose there are two points of the volume, p0 and p1 that are in the vascular system. Since V is connected there are an infinite number of smooth contours that connect p0 and p1. Some of these paths are preferable to others, and if the points p0 and p1 are in the vascular system, such a preferable path is one that follows the vascular system. If Ω is a contour connecting the points, it is often possible to parameterize Ω as a function ω:[0,s]→V such that ω(0)=p0, ω(s)=p1, and ∥ω′(•)∥=1. It is reasonable to assign for such a path a value, referred to as the path energy, by
      ∫    0    s    ⁢            C      ⁡              (                  ω          ⁡                      (            t            )                          )              ⁢                  ⅆ        t            .      It is reasonable to find such a path that has minimal path energy. A method to determine such a path in digital systems, such as a medical image, can be formulated in terms of a so-called fast marching method (see Cohen, L. and Kimmel, R. “Global Minimum for Active Contour Models: A minimal path approach.”International Journal of Computer Vision, vol. 24, No. 1, pp 57-78, August 1997).
The background information provided in the preceding paragraphs has contained special reference to the anatomy and structures of the lungs, and the localization of potential disease lesions in the lungs. The specifics of anatomical location specification will differ when the body site differs. Lesions in the human liver can be located with respect to a system of surface lobes (left, right, caudate, and quadrate anatomical lobes) based on gross surface anatomy, or a system of functional lobes and segments based on the branching structure of the hepatic and portal veins.
The embodiments of the current invention have application in the cases when anatomical locations are specified primarily with respect to the hierarchical structure of a branching tree of vessels. Examples include, in the human being, the lungs, liver, kidneys, brain, and heart myocardium. The embodiments of the invention also have application in the field of veterinary medicine, in which the anatomical structures of many vertebrate animals possess similarities to homologous structures in human beings, and hence vessel tree structures are available for location identification. The embodiments of the invention apply to anatomical vessel trees of different types, including arterial and venous blood vessel trees, lymph trees, and airway trees.
Automatic report generation is common in many “Picture Archiving and Communications System” (PACS) used in many radiology departments, such as the Carestream PACS system from Carestream Health, Inc. The report generators accumulate data from many sources, both manual and automatic and assemble the data in a standard fashion that saves time for the radiologist as well producing a report that is easy for clinicians and other radiologist to find pertinent information. A software application that identifies anatomical location containing the lesion of interest, according to a standard nomenclature system, is one of the data sources for such a system.
The embodiments of the present invention approach this problem by using methods that overcome these limitations. The invention is an ensemble of methods, where each is increasingly sophisticated, but requires more computational resources. All of these methods use the boundary of the leading edge of an expanding segmentation front that is likely to be part of the normal anatomy and finds a surface that holds this boundary fixed.